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Abstract 

We consider a collective version of Parrondo's games with probabilities 
parametrized by p £ (0, 1) in which a fraction <j> G (0, 1] of an infinite 
number of players collectively choose and individually play at each turn 
the game that yields the maximum average profit at that turn. Dim's 
and Parrondo [Optimal strategies in collective Parrondo games, Europhys. 
Lett. 63 (2003), pp. 319-325] and Van den Broeck and Cleuren [Parrondo 
games with strategy, Proceedings of the SPIE 5471 (2004), pp. 109-118] 
studied the asymptotic behavior of this greedy strategy, which corresponds 
to a piecewise-linear discrete dynamical system in a subset of the plane, 
for p — 1/3 and three choices of <j). We study its asymptotic behavior for 
all (p, <f>) € (0, 1) x (0, 1], finding that there is a globally asymptotically 
stable equilibrium if <f> < 2/3 and, typically, a unique (asymptotically 
stable) limit cycle if <f> > 2/3 ("typically" because there are rare cases 
with two limit cycles). Asymptotic stability results for > 2/3 are partly 
conjectural. 
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1 Introduction 



The Parrondo effect, in which there is a reversal in direction in some system 
parameter when two similar dynamics are combined, is the result of an under- 
lying nonlinearity. ft was first described by J. M. R. Parrondo in 1996 in the 
context of games of chance: He showed that it is possible to combine two losing 
games to produce a winning one. His idea has inspired research in such diverse 
areas as chemistry pQ, evolutionary biology [2J, population genetics [3J, finance 
0], reliability theory [5], chaos [Sj, fractals [7], epistemology [5], quantum me- 
chanics [9], and probability theory |10j . In the present paper, we analyze a 
discrete dynamical system, introduced by Dim's and Parrondo , that models 
the short-range optimization, or greedy, strategy at collective Parrondo games. 
Our analysis gives conditions under which the Parrondo effect is present. 

Let us describe Parrondo's original example. The so-called capital-dependent 
Parrondo games consist of two games, A and B. In game A, the player wins one 
unit with probability 1/2 — e, where e > is a small bias parameter, and loses 
one unit otherwise. Game B is played with two coins, the one tossed depending 
on the current capital of the player: If the player's current capital is divisible 
by 3, he tosses a "bad" coin with probability of heads po := 1/10 — e, otherwise 
he tosses a "good" coin with probability of heads p\ := 3/4 — e; he wins one 
unit with heads and loses one unit with tails. It can be shown that when e = 0, 
both games A and B are fair, hence losing when e > 0. However, the random 
mixture jA + (1 — rfB, in which game A is played with probability 7 € (0,1) 
and game B is played with probability 1 — 7, is winning for e > sufficiently 
small. Furthermore, the non-random pattern [r,s], denoting r plays of game 
A followed by s plays of game B (repeated ad infinitum), is also winning for 
e > sufficiently small, except when r = s = 1. In summary, two losing (or 
fair) games can be combined, by random mixture or nonrandom alternation, to 
create a winning game. This was the original form of Parrondo's paradox. See 
[H US HI US] for survey articles. 

Dim's and Parrondo [11] formulated a modification of the capital-dependent 
Parrondo games in which a fraction of an infinite number of players collectively 
choose and individually play the same game at each turn. They found that, 
in certain cases, by choosing the game that yields the maximum average profit 
at each turn, the surprising result is systematic losses (if e > 0), whereas a 
random or nonrandom sequence of choices yields a steady increase in average 
profit. Van den Broeck and Cleuren [16] considered also the case of a finite 
number of players. They evaluated the expected profit for this greedy strategy 
as a function of the number of players and proved that the strategy is optimal 
when the number of players is one or two but suboptimal when it is three or 
infinite. 

In this paper we consider only the case of infinitely many players, and we 
adopt the one-parameter family of capital-dependent Parrondo games of Ethier 
and Lee [T7] given by p := p 2 / (1 + p 2 ) — e and p\ := 1/(1 + p) — e with p > 0; 
the original Parrondo game B, assumed by Dim's and Parrondo [11) and Van 
den Broeck and Cleuren 16J, corresponds to p — 1/3. In order to focus on the 
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case in which two fair games produce a winning game, we assume that p G (0,1) 
for game B and e = for both games. The fraction of players who play at each 
turn, denoted here by <j> (as in [H]), was assumed to be 1/2 or 27/40 by Dim's 
and Parrondo (TT] and to be 1 by Van den Broeck and Cleuren [IB]. Here we 
let <j> range over the interval (0, 1]. 

Behrends [18 introduced a stochastic model that includes our (deterministic) 
model as a special case, and he proved that the sequence of expected (or average) 
profits is eventually quasiperiodic under certain assumptions. 

This paper develops, in the context of the Dim's-Parrondo model, techniques 
for analyzing the asymptotic behavior of a piecewise-linear discrete dynamical 
system, and may therefore be of interest even to readers unfamiliar with Par- 
rondo's paradox. 

In Section [2] we formulate a piecewise-linear discrete dynamical system for 
the capital-dependent Parrondo games played collectively according to the greedy 
strategy; it is parametrized by (p, 0) € (0, 1) x (0, 1]. In Section[3]we show that, 
in one region of the parameter space (namely, <j) < 2/3), game B is eventually 
played forever, resulting in an asymptotically fair game and, in terms of the dis- 
crete dynamical system, a globally asymptotically stable equilibrium. In Section 
[2] we show that, in the remainder of the parameter space (namely, <ft > 2/3), 
there is an initial state that yields a periodic pattern of games, resulting in an 
asymptotically winning game and, in terms of the discrete dynamical system, a 
limit cycle. In fact, in a very small region of the parameter space, there are two 
limit cycles. Section [5] attempts to show that (again assuming </> > 2/3), where 
there is a unique limit cycle it is asymptotically stable (in fact, it is globally 
asymptotically stable unless there is an unstable equilibrium). The proofs of 
the assertions in Section [5] are incomplete, so some of our findings are stated as 
conjectures. In Section [6j we confirm the assertions just made concerning the 
asymptotic profitability of the greedy strategy. 

For example, if (p, (f>) — (1/3, 1/2) (and e = 0), our results show that there is 
a globally asymptotically stable equilibrium with game B eventually played for- 
ever. This is contrary to a computational result of Dims and Parrondo [TT], who 
found the pattern [1,40] in this case (i.e., AB 40 AB 40 AB 40 ■■■). The anomaly 
is likely attributable to roundoff error; 64-bit arithmetic (C++) is insufficient 
here. 

Let us introduce some additional notation. As defined in the second para- 
graph above, the game pattern [1, 2] stands for ABB ABB ABB ■ ■ ■ . It will be 
useful to have a concise notation for game sequences that are eventually peri- 
odic. We write AB ABB ABB ABB • • • , for example, as ABABB, just as one 
would write the binary expansion of the fraction 5/14 as 0.01011. 

With this notation, we can describe the limit cycles that appear in terms 
of the game patterns. They are of two types, either one of AB 2 , AB 4 , AB 6 , 
. . . , or one of AB 4 AB 2 , AB 6 AB 4 , AB 8 AB 6 , .... For further simplicity, we will 
also denote these patterns by [1, 2], [1, 4], [1,6], . . . , and by [1, 4, 1, 2], [1, 6, 1, 4], 
[1,8,1,6], .... 
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2 Preliminaries 



It is well known that a Markov chain {X„}„>o with state space {0, 1, 2} underlies 
the capital-dependent Parrondo games; here X n represents the player's capital 
modulo 3 after n games. When playing game B it evolves according to the 
one-step transition matrix 

/ po 1 - p \ 

P°B ■= [ I-Pl Pl , (1) 

V pi i-pi o / 

where po := p 2 /(l + p 2 ) and p\ := 1/(1 + p) with < p < 1. When playing 
game A it evolves according to the one-step transition matrix P£ defined by 
the matrix in (JlJ with p — 1 (i.e., with po = Pi := 1/2). The unique stationary 
distribution tt :— (ttq, tt±, tt 2 ) of Pg is given by 

1+P 2 p(l + p) l + p 

770 2(1 + p + p 2 )' 771 2(1 + p + p 2 )' ^ 2(1 + p + p 2 )' 

while that of P% is (1/3, 1/3, 1/3). 

Given < <fi < 1, consider a large number TV of players, of whom (j)N are 
selected at random. Everyone in the sample of size <f>N independently plays 
game A or everyone independently plays game B, the choice determined by the 
strategy. When playing game B, each player uses his own capital to determine 
which coin to toss. Let xo(ra) be the fraction of the players whose capital is 
divisible by 3 after n turns. If the players in the sample collectively choose 
and individually play game B, then the expected average profit, conditioned on 
xo(n), is equal to 

x (n)(2po - 1) + [1 - x (n)](2pi - 1) = -x (n) - 9 + [1 - x {n)}^-— -, 

1 + p A l + p 

which is nonpositive if and only if Xo(n) > 7To. Game A always has expected 
average profit equal to 0. So the strategy of maximizing the expected average 
profit at each turn can be summarized by the rules "play game A if Xo(n) > ttq" 
and "play game B if Xo{ri) < 7r ." In particular, if both games have expected 
average profit equal to 0, then game A is played. 

We investigate the mean-field limit as N — > 00, in which case the model 
is deterministic. (We will not try to justify this; the preceding paragraph was 
included mainly for motivation.) Let Xi represent the fraction of players whose 
capital is congruent to i (mod 3) for i = 0, 1, 2. Then xq + x\ + X2 = 1. Thus, 
in the state space defined by 

A := {{x , xi,x 2 ) : x > 0, x\ > 0, x 2 > 0, x + x\ + x 2 = 1} 

we have a discrete dynamical system given by 

(xo(n + l),xi(n + l),x 2 {n + l j) = F(xo(n), Xi(n), x 2 (n)), n > 0, 
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Table 1: Dependence on of the nonunit eigenvalues e\ and e 2 of Pb- 
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where 

„/ v J (xq,x 1 ,x 2 )Pa if^o>7To, 

F{x ,xi,x 2 ) := < 

[(x ,X 1 ,X2)Pb II Xq < 7T , 

and Pa := (1 — 4)1 + <t>P% and P B := (1 - 0)/ + </>P£. Clearly, the function F 
is piecewise linear but discontinuous. 

Let us define the projection p of A onto {(a;o,xi) : xq > 0, x\ > 0, xq + xi < 
1} by p(xo,x\,X2) '■= (xq,xi). It is a one-to-one transformation. While the 
trajectory belongs to A, it will often be convenient to regard it as belonging 
to p(A), a subset of the plane. In fact, we could redefine F in terms of 2 x 2 
matrices, but this does not appear to simplify matters. 

To study the asymptotic behavior of the system, we will need the spectral 
representation for matrices Pa and Pb. We note that 7r is also the unique 
stationary distribution of Pg. The nonunit eigenvalues of Pb are given by 
et := l-4+4el and e 2 := l-0+0e^, where, with S := + p 2 ){l + Ap + p 2 ), 

1 (l-p)S , „ 1 



ei := — — + ^ — — and e 9 := — — — 



2 ' 2(l + p)(l + p2) ^ ' 2 2(l + p)(l + p2) 

are the nonunit eigenvalues of P B . Observe that e\ = if <f> = 4%t an( i e 2 = if 
= 0i, where 

1 2(l + p)(l + p 2 ) 



-2 



3(l + p)(l + p 2 ) + (l- P )5' 

2\ 



1 2 (l + p)(l + p 2 ) 



1-eJ 3(l + p)(l + p 2 )-(l-p)5- 



(2) 



Since > e\ > -1/2 > e° 2 > -1 for < p < 1, we have 1/2 < 0i < 2/3 < 
02 < 1 for < p < 1. Because the nonunit eigenvalues ei and e 2 will play an 
important role in what follows, we indicate their dependence on in Table [T] 

We define the diagonal matrix D := diag(l, e\, e 2 )- Corresponding right 
eigenvectors (both for Pb and P B ) are 

(l + p)(l-p 2 -S) 
2 + p + 2p 2 + p 3 + pS 
-(l + 2p + p 2 + 2p 3 - S) 









... 




--( 
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(1 + P )(l- P 2 + S) 
r 2 := | 2 + p + 2p 2 + p 3 - pS 

-(l + 2p + p 2 + 2p 3 + S) j 

They are linearly independent, so we define R := (ro,r 1 ,r 2 ) and L := J? -1 . 
The rows of L are left eigenvectors, and the spectral representation gives 

Pg = n > 0. (3) 

Of course, Pa is the special case p = 1 of Pg, so it follows from ([3]) (or a 
simple induction argument) that 




PI = I <l„ 1 - 2J„ +, I . ;/ > II. 

where d„ := [1 - (1 - 30/2)™] /3. 

3 A globally asymptotically stable equilibrium 

for (j) < 2/3 

In this section we show that, corresponding to playing game B forever under 
the greedy strategy there is a globally asymptotically stable equilibrium when 
<j) < 2/3 and an unstable equilibrium when 2/3 < <f> < <f>2- Let 

:= {(xq,xi,x 2 ) € A : x Q > tt } (resp., A B := A - A^) 

be the set of states at which game A (resp., game B) is chosen under the greedy 
strategy. In fact, it will be useful to extend this notation considerably. For 
example, A abb a is the subset of A^ such that the first four games played are 
ABBA (in that order), and A BB ab * s * ne subset of such that the complete 
game sequence is BBAB. 

Proposition 1. Under the greedy strategy, game A is chosen for only finitely 
many consecutive turns, given an initial state in A a- If 2/3 < (j) 1; then, 
after only one play of game A, game B is played. 

Proof. After n plays of game A, the initial state, say (xo, Xi, x%) € A^, moves 
to 



(x ,xi,x 2 ) 



111 

3' 3' 3 



(4) 



If < 4> < 2/3, the trajectory converges to the limit (1/3,1/3,1/3) along the 
line segment that connects the initial state (xq, Xi, x%) with (1/3,1/3,1/3) as 
n — > 00. Since 1/3 < ttq for < p < 1, we see that, after a finite number of 
consecutive plays of game A, game B is played. If2/3 < </> < 1, then we have 

(xo,Xi,S 2 )P j4 (l,0,0) T = l+(l-lA(*0-l)<l< 7T0, 
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which means that, after only one play of game A (requiring xq > ttq > 1/3), 
game B is played. □ 

Theorem 2. If < <fi < 2/3, wherever the initial state is located, the greedy 
strategy chooses game B forever except for an initial finite number of turns. 
In particular, the discrete dynamical system has a globally asymptotically stable 
equilibrium, namely 7r, the stationary distribution of Pb- If 2/3 < <fi < 1, game 
B is chosen forever only when <f> < (f>2, where (f>2 is defined by |1J), and only 
when the initial state belongs to one of at most four one- dimensional regions, 
which will be specified below in terms of p and <f>. 

Proof. Recall that A-g denotes the set of initial states from which game B is 
played forever. Once we know that Ajj is eventually reached from any initial 
state, we have a linear discrete dynamical system 

(x Q (n + l),x 1 (n + l),x 2 (n + 1)) = (x (n), Xi(n), x 2 (n))P B , n > 0, 

or (xo(n), X\{n), X2{n)) = (xq, X\, X2)P B l for each n > 1, with (xq, xi, X2) G A^r. 
But since Pb is irreducible and aperiodic, Pg -+ II, where II is the 3x3 matrix 
with each row equal to tv. This implies that (xo(n), xi(n), x 2 (n)) — > 7r for all 
(xo,xi,x 2 ) G Ag-, and this leads to the global asymptotic stability. 

Since the trajectory enters Ab eventually by Proposition [T] it is enough to 
consider a trajectory that starts from A^. Let (xq, x\, x%) € As and let 7To(n) 
be the fraction of the players whose capital is divisible by 3 after n plays of 
game B. Then 7Tq(0) = xq < ttq. From the spectral representation we have 



(x ,xi,l - x - x 1 )RD n L(l,Q,0) T 



2,g 



7T (n) = (x ,xi,x 2 )P£(l,0,0) T 
4(1 + p + p^)b 

- e?[2x (l + P + P 2 )(l + P 2 - S) + 4x x (l + p + p 2 )(l + p 2 ) 

-(l + 2p + 3p 2 -S)(l + p 2 )} 
+ e^[2x (l + p + p 2 )(l + p 2 + S) + 4x 1 (l + p + p 2 )(l + p 2 ) 

-(l + 2p + 3p 2 + S)(l + p 2 )]}, 



from which it follows that 



7T - TT (n) = cie™ - c 2 e 



2 • 



(5) 



where 



ci := 



('2 



2 r 



1 + P 
2S 

1 + P 2 r 
25 



1 - 
1 
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1 + P 2 

S 
1 + p 



(Xq - TTq) + 2(.Tl - 7Ti) 
2 ( (l - TTo) + 2(.Tl - 7Tl) 



Notice that c\ > c 2 for xq < n and < p < 1. Also, S/{1 + p 2 ) > 1 for 
<p < 1. 
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The region A-g depends on the nonunit eigenvalues of Pb, so we derive it 
separately in the seven cases of Table [I] as follows. 

Case 1. < <f> < fa. Since e\ > e 2 > 0, if c\ > we have 7r (n) < 7r for 
all n > 1. On the other hand, if Cj < 0, we can find a positive integer n such 
that (e2/ei) n < Ci/c 2 < (e2/ei)™ _1 , and we conclude that 7r (n) > tt . Thus, it 
follows that 

Ag = l^(x ,x 1 ,x 2 ) e A B : ^1 - ^ ( x ° - ^o) + 2 ( Xl ~ 71-1 ) - 

For (xo, %i, x%) <E Ab — A-g, we have xq < ttq, hence X\ < Therefore, 
(xo, x\, £2) moves to state (j/OjJ/iiJ/a) : = O^Oi X2)Pb> which satisfies 

fal - (2 + p)z - (1 - /o)x x ] 0[1 - (2 + p)ir - (1 - p)7n] 

2/o - a;o = 7- > r- = 

1 + p 1 + p 

and 

0[p(l + p 2 ) - p(l - p)x Q - (1 + 2p)(l + p 2 ) Xl ] 



Vi-%1 = 



> 4>w + p 2 ) - pq - p)^q - (1 + 2p)(i + p 2 )tti] = 

Therefore, in the region Ab — A-g, as long as game B is played, the trajectory 
continues to move in the positive xo and x\ directions and finally reaches the 
region A a- (It cannot reach Ag- first, and it cannot remain in A^ — A-g forever, 
for in cither case it would have started in A^-.) Once it arrives at a certain 
state (j/o, V 1,1/2) € A a, it moves along the line segment between (y , 2/1,2/2) and 
(1/3, 1/3, 1/3) € A-g, proceeding 30/2 of the way (see Q). After one or more 
such jumps from A^, the trajectory will either reach A-g or return to — Ag. 
Even if it returns to As — Ag, it eventually enters Ag because it keeps moving 
in the positive x\ direction while it visits the two regions Ab — Ag and A^, 
and from A^ it moves in the positive X\ direction 30/2 of the way toward 1/3, 
ensuring that this alternation between Ab — A^g and A^ cannot go on forever. 
See Figure [TJ 

Case 2. = fa. Here e\ > = e 2 . Consequently, no{n) < no for all n > 1 if 
and only if ci > 0, so that 

Ag = l(x ,xi,x 2 ) 6 Ab ■ (l — 2 ) (x Q - n Q ) + 2{x\ - tti) > 



For (x , xi, x 2 ) € Ab — A-g, since c\ < we have 7r (l) > n . So game A is 
played after one play of game B. Therefore, when the trajectory starts in the 
region - A-g, after a few alternations of game A and game B, it enters the 
region Ag and stays there forever, just as in Case 1. 

Case 3. fa < (f) < 2/3. In this case, e\ > > e 2 and |ei| > \e 2 \. We claim 
that a necessary and sufficient condition for iro(n) < ttq for every n > 1 is that 
c\e\ — c 2 e 2 > 0. It is clearly necessary (take n = 1 in (kmI). For sufficiency, note 
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0.0 0.2 0.4 0.6 0.8 1.0 ° 0.0 0.2 0.4 0.6 0.8 1.0 



Figure 1: Four of the seven cases of Theorem [2] illustrated with p = 1/3. 
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that ciei — > implies c± > 0. Thus, we have cie" — 0262 > for all n > 1 
if in addition c\ > c 2 > 0. If c\ > > C2, then for even n > 2, cie™ — 0263 > 0, 
and for odd n > 1, 



cie x - c 2 e 2 = 



C2 / e 2 



ci \ ei 



> Cl ef(l- — |>0, 
ciei 



proving the claim. It follows that 



A 5 = 



(x ,xi,a;2) e A B : 

(30-2)(l + p) 



1 



(1-P) 



(X - 7T ) + 2(^1 - 7Ti) > 



(6) 



Let (xo,xi,X2) G As — A-g. Since ciei — C2e2 < 0, game A is played after 
one play of game B. Especially if x\ < tti, the trajectory enters A-g after a few 
alternations of game B and game A as in Case 1. If x\ > tti, it moves to state 
(yo,2/i,J/2) := (x ,xi,x 2 )Pb in A^, which satisfies 

2/x - tti = [2(1 + + p 2 )(l + p + p 2 )]~ 1 {[20(l + p + p 2 ) - (1 + p)>(l + p 2 ) 

- 20p(l - p 3 )x + 2[1 + p - 0(1 + 2p)](l + p 2 )(l + p + p 2 )xi} 

> [2(1 + p)(l + p 2 )(l + p + p 2 )]- 1 ^! + p + p 2 ) - (1 + p)>(l + p 2 ) 

- 20p(l - p 3 )^ + 2[1 + p - 0(1 + 2p)](l + p 2 )(l + p + p 2 )^} 

= 0, 

where the inequality uses 4> < 2/3 < (l + p)/(l + 2p) for < p < 1. From Propo- 
sitionjl it follows that the trajectory converges to the limit (1/3, 1/3, 1/3) € A-g 
along the line segment between (2/0,2/1,2/2) € A^ with yi > iri and (1/3,1/3, 
1/3). Since this line is above (in p(A)) the critical line segment for Ag in (JH]), 
after a finite number of plays of game A the trajectory directly enters the region 
Ag. See Figure [1] 

Case 4. (f> = 2/3. Since e\ > and e2 = — ei, for even n > 2, ttq — TTo(n) = 
e? (to — £0) > 0, an d for odd n > 1, 



7T - 7r (n) = (1 + p 2 )e"[a; - ir + 2(x ± - Ki)]/S. 



(7) 



So we have 



A5 = {{X , Xl,X 2 ) £ A B ■ X - 7T + 2(X1 - 7Tl) > 0}. 

If (xq, Xi : x 2 ) €E As — A-g, since 7r (l) > 7r by Q, game A is played after 
one play of game B. But after playing game A the trajectory directly moves 
to (1/3, 1/3, 1/3) G Ag because Pa is the 3x3 matrix each of whose entries 
is 1/3. Therefore, there are only three possible sequences of games, namely B, 
AB, and BAB. See Figure [lj 
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Case 5. 2/3 < <j) < fa. Here e\ > > e 2 and | e 1 1 < \e%\. If C2 = 0, then 
because c\ > c 2 = we have 7To(n) < 7To for all n > 1. On the other hand, if 
c 2 7^ 0, then 7r (n) > 7r for some n> 1. This gives 



A s = \ (xa,x 1 ,x 2 ) e A B : 1 



5 



1 + P 2 



(lE - 7T ) + 2(jCl - 7Ti) = 



(8) 



We recall from Proposition [T] that there is only one play of game A when the 
trajectory enters Aa- Since (wq,wi,W2) € moves to 



(x ,x 1 ,x 2 ) := {wo,w 1 ,w 2 )Pa = ^<f> 
any state (wq, W\, w^) € A^ satisfying 



1 1 1 

3' 3' 3 



+ 1 - ( w o,wi,u;2), 



1 + p 2 



= o 



moves to Ag- after one play of game A. Defining 



f{x) :=-- 1 + 



S 



1 



■ - 27n)(l + p 2 ) + {fa~ 27r )(l + p 2 + 5) 
2(30-2)(l + p 2 ) 



for 7Tq < x < 1, we have 



AB 



{(w ,wi,w 2 ) € Aa : iui = /(io )}. 



(9) 



Since the slope m oi y = f(x) satisfies m < —1, we need both /(7To) > and 
/(I) < to ensure that A A g is nonempty. For < p < 1 and 2/3 < <fi < 4> 2 it 
is easy to check that f(no) > 0. Next we have 



/(I) = 



(1 + p)[(l - p)(l + P 2 ) + (1 + p)5] - 20(1 + p + p 2 )5 



2(30-2)(l + p 2 )(l + p + p 2 ) 
So it follows that if 2/3 < <f> < fa with 

(l + p)[(l-p)(l + p 2 ) + (l + p)S] 



2(1 + p + p 2 )S 

then /(l) > and A^ = 0, whereas if 03 < <f> < fa, then A A -^ 7^ £ 
latter occurs only in regions 5-8 of Figure [2] in Section [5j See Figure [I] 
Similar arguments show that 



and 
where 



A bab = {( w o,wi,w 2 ) £ A B '■ W\ — g(w )} 

A BBAB = {(W ,W 1 ,W2) G A B '. W\ = h(w )}, 



The 

(10) 
(11) 



9(x) 



1 



5' 



.9i (p» 



(30-2). 92 (p,0)' 
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gi(j>, 0) := 0(30 - 2)[(1 + 2p)(l + p 2 ) + S] - (1 + p)[2(0 - 27n)(l + p 2 ) + 
2tto)(1 + p 2 + 5)], 52 (P, 4>) := (30 - 2)(1 + p)(l + p 2 ) + 0(1 - p)S, 



2 V 1 + P 2 y (30-2)/i 2 (p,0)' 

hi(p, 0) := 0(30 - 2){2(1 + p)[(l + 2p)(l + p 2 ) + 5] - 0[2 + 6p + 3p 2 + 4p 3 + 
3p 4 + (2 + 2p - p 2 )S}} - (1 + p) 2 [2(0 - 2tti)(1 + p 2 ) + (0 - 2tt )(1 + p 2 + 5)], 
and h 2 {p,4>) := -2(1 + p) 2 (l + p 2 ) + 60(1 + p) 2 (l + p 2 ) - 2 (5 + 10p + 6p 2 + 



10p 3 + 5p 4 ) — 0(30 — 2)(1 — p 2 )S. Further, ( 10 ) is nonempty in a region slightly 
smaller than the union of regions 7 and 8 of Figure [2] in Section [5j while ( 11 ) is 
nonempty only in a very small subset of region 8 of that figure. 

Finally, it can be shown that &abab = ' ^-abbab = > ano - ^bbbaTS = 
0, implying that there are no other ways in which game B is played forever. In 
summary, in the case of 2/3 < < 02, only when the initial state belongs to 
p]), (JoJ) , (10), or (11), some of which may be empty, is game B played forever. 

Cases 6 and 7. 02 < < 1. Here e2 < e\ < 0. Clearly, A-g = in these 
cases. □ 



4 A limit cycle for > 2/3 

In this section we show that, whenever > 2/3, there is at least one limit cycle, 
at least when the discrete dynamical system starts from a certain initial state, 
which will be specified. The game patterns that occur include [1, n], denoting 1 
play of game A followed by n plays of game B, for even n > 2, and [1, n, 1, n— 2], 
denoting 1 play of game A, n plays of game B, 1 play of game A, and n — 2 
plays of game B, for even n > 4. The value of n depends on p and 0. 

We will need three lemmas to prepare for the next theorem. The proofs are 
trivial and therefore omitted. 

Lemma 3. If 1 < a < b and c > 0, then (a n + c)/(b n + c) is decreasing in 
n > 1. 

Lemma 4. If0<a<b<l, a + b<l, and c > 1, then (c — a n )/{c— b n ) is 
decreasing in n > 1. 

Lemma 5. If < c < |, i/ie functions 

/(x) := and qtx) := — 

x ' l-cx n w l-cx^ 1 

are increasing on (0, 1) for each n > 1. 

We will also need the functions 

:= 0(1 - p){e 2 l [2 + e?(30 - 2)] [3(1 + p)(l + p 2 ) - (1 - p)5] 

- e?[2 + e£(30 - 2)][3(1 + p)(l + p 2 ) + (1 - p)S]}/2, (12) 
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E n , m ■= 0(1 ~ p){<$[2 + e?(30 - 2)] [3(1 + p)(l + p 2 ) - (1 - 

- er[2 + ^(30 - 2)][3(1 + p)(l + p 2 ) + (1 - p)S]}/2, (13) 

G n , m := 0(1 - p){e%[4 - e 2( "~ 1) (30 - 2) 2 ][2 - e r 2 (30 - 2)][3(1 + p)(l + p 2 ) 

(1 - p)S] - e?*[4 - e 2( - 1) (30 - 2) 2 ][2 - 6^(30 - 2)] 
•[3(l + p)(l + p 2 ) + (l- P )5]}, (14) 

H n , m := 0(1 - p){4"[4 - e ? ( ^ 1} (30 - 2) 2 ][2 - e£(30 - 2)][3(1 + p)(l + p 2 ) 

- (1 - p)S] - e?{4 - e 2( "- 1} (30 - 2) 2 ][2 - e»(30 - 2)] 

•[3(l + p)(l + p 2 ) + (l-p)S]}. (15) 

The significance of these functions is explained in the following theorem. 

Theorem 6. For even n > 4, the implicitly defined curves G n>n -2 — 0, E n _2 — 
0, £ , rlj , l -2 = 0, H n ,n-i = 0, and G n +2,n = are monotonically ordered, from 
highest to lowest, in {(p,<p) € (0,1) x (2/3,3/4) : <f> < 2 }. More precisely, 

G„,„_2 = is above £„_ 2 = 0, (16) 

E n - 2 = is above S n> „_ 2 = 0, (17) 

E n , n -2 = is above H n , n _ 2 = 0, (18) 

H n ,n-2 = is above G n+2 ,„ = 0, (19) 

for n = 4, 6, 8, . . .. Furthermore, the functions defining the curves are positive 
above, and negative below, the curves. 

For even n > 2 and (p, </>) S (0, 1) x (2/3, 1], the greedy strategy leads to the 
periodic pattern [l,n] starting from the corresponding stationary distribution as 
the initial state if and only if E n ^ n ^ 2 < and E n > 0. (E 2 fl < is automatically 
satisfied.) 

For even n > 4 and (p, 0) G (0,1) x (2/3,1], the greedy strategy leads to 
the periodic pattern [l,n, l,n — 2] starting from the corresponding stationary 
distribution as the initial state if and only i/G„ in _ 2 < and H n n _ 2 > 0. 

Remark. In particular, if > 2/3, there is at least one limit cycle. 



If (p, 4>) belongs to the region between the curves in ( 16 1, there arc two limit 



cycles, of the forms [1, n, 1, n — 2] and [1, n — 2]. If (p, 0) belongs to the region 



between the curves in (17 1, there is one limit cycle, of the form [l,n, l,n— 2]. 



If (p, 0) belongs to the region between the curves in (18 1, there are two limit 
cycles, of the forms 1, n — 2] and [l,n]. If (p, 0) belongs to the region 
between the curves in ( |19[ ), there is one limit cycle, of the form [l,n]. 

The regions with two limit cycles are very small. See Table [2] for the case 
p=l/3. 

Proof. Let TT[i >n ] be the stationary distribution of PaPq- Then we have 

(7r [1>n] - 70(1,0, 0) T = E n /D n , (20) 
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Table 2: Critical 0-values separating regions at p = 1/3. Numbers are truncated 
(not rounded) at 18 decimal places. 



form of 



lower-boundary 
equation 



4>- value 
at p — 1/3 



limit cycles 



[1,2] 
[1,4,1,2], [1,2] 

[1,4,1,2] 
[1,4,1,2], [1,4] 

[1,4] 
[1,6,1,4], [1,4] 

[1,6,1,4] 
[1,6,1,4], [1,6] 
[1,6] 



G4,2 = 

E 2 = 
£4,2 = 

#4,2 = 

G 6 ,4 = 

£4 = 

E 6A = 

#6,4 = 
G8,6 = 



0.688 066 413 565 052 628 
0.688 066 239 503 137641 
0.688 026 898 650 299426 
0.688 026 881018 074 821 
0.677 218 563 694 275 305 
0.677218 563 614 298 209 
0.677217 953 395 292 912 
0.677217 953 388 847194 
0.673 669128 225 600 196 




(21) 



Now D n is positive for all positive integers n because < 30 — 2 < 1 and |ei | < 
|e 2 | < 1. Notingthat e 1 +e 2 = -(30-2), e 2 -e x = -0(1- p)S/[(l + p)(l + p 2 )], 
and eie 2 = 1 - 30 + 20 2 (1 + p + p 2 ) 2 /[(l + p) 2 (l + p 2 )], we have 



where g(p, 0) := 5 i(0)(l + 4p + 4p 7 + p 8 ) + 52 (0)p 2 (l + p 4 ) + <? 3 (0)p 3 (l + p 2 ) + 
£ 4 (0)p 4 with 5l (0) := -15 + 480-630 2 +440 3 -120 4 , 52 (0) := -120 + 3960- 
54O0 2 + 4O40 3 - 12O0 4 , ff3 (0) := -180 + 6000 - 8280 2 + 6320 3 - 1920 4 , and 
54 (0) := -210 + 6960 - 9540 2 + 7280 3 - 2280 4 . Since the functions g x , g 2l g 3 , 
and gi are increasing on (0, 1], we have g{p, 0) > g(p, 2 ) for 2 < < 1 and 



where g 5 (p) := 17 + 68p + 80p 2 + 92p 3 + 134p 4 + 92p 5 + 80p 6 + 68p 7 + 17p 8 - 
3(1 - p 2 )(5 + lOp + 6p 2 + 10p 3 + Sp 4 )^. Using 

[17 + 68p + 80p 2 + 92p 3 + 134p 4 + 92p 5 + 80p 6 + 68p 7 + 17p 8 ] 2 

- [3(1 - p 2 )(5 + lOp + 6p 2 + 10p 3 + dp^S} 2 = 64(1 + p + p 2 ) 8 > 0, 

we have g$ (p) > and hence g(p, 2 ) > for < p < 1. It follows that if 
02 < < 1, then E 2 > 0, which implies that 7T[i, 2 ] <E A4. 



6 2 (l-p) 2 g (p,0)5 
(l + p) 4 (l + p 2 ) 2 ' 



g(p, fa) 



16(l-p)(l + p) 3 (l + p 2 ) 3 g 5 (p)S 
[3(l + p)(l + p 2 )-(l-p)5] 4 
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More generally, a similar argument shows that 



(*■[!,«] -P 



7r)(l,0,0) T = E ntm /D n , 



(22) 



where E n>m and D n are as in (131 and (21). Notice that E, 
E 2 .o < by Proposition [Tj Next, 



E n . First, 



E2,l — 



(l+ P ) 2 (l+p 2 ) 



[h{4>){! + 2p + 2p A + p 4 ) - 6(1 - 0)(5 - 90 + 90 )/? ], 



-15 + 4O0 - 450 2 



Since h is increasing, its maximum on 



-2, so we have i?2,i < 0, which proves that tt^ 12 ]PaPb £ Ag 



where h((f>) :— 
(0,1] is h(l) = 

We need one more play of game B to return to tv^^PaP'b = 7r [i,2] € Ay 
Hence if 02 < < 1, the trajectory that starts from initial state ttm 2 ] € A . 
follows the periodic pattern [1,2] under the greedy strategy. 

Next consider the case of 2/3 < < 2 . We can rewrite E n in (12) as 



(1-P) 



[3(1 + p)(l + p 2 ) + (1 - p)S]e 2 l [2 + e?(30 - 2)] 



where F„ := (ei/e 2 ) n [2 + e 2 l (30-2)]/[2 + ei l (30-2)]. In this case, since e 2 < < 
ei with |e 2 | > |ei|, we find, for all odd n > 1, that F n < and therefore i?„ < 0. 
Moreover, F n > for all even n > 2, and the sequence {F n : n — 2,4, 6, . . .} 
is decreasing to since F n = [(l/e 2 )" + (30 - 2)/2]/[(l/ei) n + (30 - 2)/2] and 
therefore Lemma [3] applies. Thus, we let s denote the smallest even n > 2 such 
that < F n < 0i/0 2 . Equivalently, s := min{n € {2, 4, 6, . . .} : E n > 0}, hence 
E s > and (if s > 4) £ s _ 2 < 0. By Q, 7r [M] e A A . 
For odd m < s, we have E„ m < OTTf 



E., 



< 0, then E„ m < for all even 



m < s — 2 and since 7Tn s iPaP| = 7rri )S ]) we can conclude that after s plays of 
game B, the trajectory returns to the initial state 7rr ljS i € A A . Hence for (p, 0) 
satisfying E SiS -2 < (with s defined as above), the trajectory that starts from 
the initial state T[i, s ] follows the periodic pattern [1, s] under the greedy strategy. 
(Notice that if s = 2, then E ss _ 2 = i?2,o < automatically.) Moreover, the 
conclusion fails if i?„ < (implying 7T[i )n i G A s ) or if £ , „ i „_ 2 > (implying 



^[i,u]PaPb 
the theorem. 

We next claim that for (p, 0) satisfying E SJS _ 2 > with s > 4, the greedy 
strategy leads to periodic pattern [1, s, 1, s — 2] if we start from the initial state 
7r [i.s,i.s-2] £ A A . Calculations similar to (22) give 



€ A A ). This proves the assertions in the second paragraph of 



(^[l,n,l,n-2]PAPB ~ 7i ")(1j 0) 0) T = G n ^ n /I n , 

(7v lhn ^2]PAPSPAP 7 B n - *■)(!, 0, 0) T = H n>m /I n , 



where G n ^ m and H n ^ m are as in (14) and (15) and 

I n := 2[2 - e' 1 1 - 1 (30 - 2)] [2 - e 2 l - 1 (30 - 2)]£>„_i. 
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Since < 30 — 2 < 1 and |ei| < | e2 1 < 1, we have /„ > 0. To prove the claim 
we need to show that G s ^ m < for < m < s — 1 and G s , s > 0, as well as 
iJ SjTO <0for0<m<s — 3 and H SyS - 2 > 0. Since e 2 < < ei with | e2 1 > |ei|, 
it is sufficient to prove that G s s _2 < 0, G SJS > 0, H s < 0, and H s s _ 2 > 0. 
From the fact that E s _ 2 < we have 



- > 



2 + e^ 2 (30-2) 
2 + e s 2 - 2 (3c/)-2)J \(t> 

4-e 2(s ~ 2) (30-2) 2 \ /2-er 2 (30-2)\ /0 



2(s-2) 



(30-2)' 



„s-2 



(3^-2); \^ 



(23) 



Here we can show that the sequence (4 
decreasing in m if we apply Lemma [4] with 



e 2m (30-2) 2 )/(4 



l (30-2) 2 ) is 



b = e%, and c = 4/(30-2)^ 



where we recall that 2/3<0<02<l,soO<a<6<l (see TableJT]) and c > 4. 
It remains to show that a+b < 1. Let us write e\ = 1 — 0+0eJ — 1 — (3/2)0+0e° 
and e 2 = l-(f> + (j>e 2 = 1- (3/2)0-0e°, where e° := (1- p)S/ [2(1 + p) (I + p 2 )} G 
(0,1/2). Then 



'1 -r e 2 
2 i 



2 

6^ + 5</> 



I- 3 , 
2 s 



since 2/3 < < 02 < 1. 



Hence from ( 23 ) we have 

2(s-l) 



+ 20 2 (e°) 2 < 2 1- -0 + 



1 + (1-0)(1 



< 1 



> 



4- ei — ; (30-2) 2 \ / 2- e ^(30-2) 
4 - e5 ( * _1) (3^ - 2)2/ V2 - e s r 2 (30 - 2) 



which is equivalent to G StS - 2 < 0. By the same reasoning, for even n > 4, 
< implies G n>n - 2 < 0, which yields (16 1. 
We also have 



> 



> 



ei 

4-«I 



e2 
ei 

2(s-l) 



4-e 2(s - 1) (30-2) 2 
4-e 2(s ~ 1) (30-2) 2 



(30- 2) 2 ^ ( 2~e s 2 -\i(j)~2) 
2-er 2 (30-2) 

2-6^(30-2) 



4-e 



2( s -l) 



(30-2)2 



(24) 



which is equivalent to H s s 



2e 2 



2-ef(30-2) 
-4 < 0. Notice that the last inequality in (24) uses 
e|(30-2) 2-e^(30-2) 



2e\ - ef(30- 2) 



> 



e?(30-2) : 



(25) 
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which is equivalent to 



2 e j-eS(30-2) 
2-e|(30-2) 



> 



2e\ 



e!(30-2) 



ef(30-2) 



for even s > 4. To confirm the latter inequality, divide both numerators and 
denominators by 2 and apply Lemma [5| 



that 



On the other hand, from E ss _ 2 > and the argument below (23), it follows 

ef(30-2)\ 

-2)JUJ 

e|(30-2) x 



- < 



< 



4(30-2); 
e 2s (30-2) 2 



(30 -2) 2 / V2-ef(30-2) / \0 



e 



2(s-l), 



(30-2) 2 \ /2-ef(30-2)\ ( <j> 



4-e 



2( S -1), 



(3^-2)2/ V2-ef(30-2)7 V0: 



(26) 



which is equivalent to H s s _2 > 0. By the same reasoning, for even n > 4, 
E n ,n-2 > implies H n ^ 2 > 0, which yields (18). 
Similarly, we have 



< 



< 



ei (s ~ 1} (30-2) 2 \ /2-e|(30-2) 



4-e 



2( S -1) 



(3^-2)5 
ei ( ^ 1} (30-2) 5 



2(^-1), 



2-ef(30-2); V0 



2-er 2 (30-2) 



(30 -2) 2 / V2-epW-2) 



(27) 



where the last inequality follows from (25), and this is equivalent to G s s > 



This almost proves the assertions in the third paragraph of the theorem, except 
that we have implicitly assumed that E n>n _2 > and E n _2 < 0, both of which 
are stronger than necessary. We can weaken the former to H nn - 2 > 0, in which 



case (26) is no longer necessary and (27) follows as before. We can weaken the 



latter to G„.„_ 2 < 0, in which case (23) is no longer necessary and (24) follows 
as before. (When E n _ 2 > and G„ jn _ 2 < 0, we have s = n — 2, so we apply 
the inequalities involving s with s replaced by n.) Finally, the necessity of the 
inequalities is clear: If G n ,n-i > 0, then 7T[i ! „ > i > „_2]Pa-F > b _ ^A, while if 



El n , n -i < 0, then jn ,i, n _2] — 7T[ 



P A P5P A P» G A, 



For (17), it is enough to show that, for even n > 4, E n ^ n - 2 — -En-2 > for 



all (p, 0) G (0, 1) x (2/3, 3/4). This will then imply that, for even n > 4, when 
£' n _2 =0we have E nn _2 > 0, which yields (17). 



Now 



E n , n _ 2 - E n _ 2 = eJ-'eTVW - 2 )(! - P) 2 8[6(1 + P) (1 + P ) 



- 0(7 + 14p + 12p 2 + 14p 3 + 7p 4 )]/[(l + p) 2 (l + p 2 )] 
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which has the sign of 6(l+p) 2 (l+p 2 )-0(7+14p+12p 2 + 14p 3 +7p 4 ). The latter 
is decreasing in <f> and, at <j> = 3/4, equals (3/4) (1 + 2p + Ap 2 + 2p 3 + p A ) > 0, 
hence it is positive in (0, 1) x (2/3,3/4). 



For (191, it is enough to show that, for even n > 4, if H n n _ 2 — 0, then 



G n +2,n > 0. Equivalently, it suffices to show that, if 

^ e 2 \ 2 ( 4 - e\ n ~ 2 (30 - 2) 2 \( 2 - eg (30 - 2) 

then 



ex) V4 - e 2 ™" 2 ^ -2) 2 )\2- e?(30 - 2) 



e 2 W4- ef +2 (30 - 2) 2 \ /2 - e£(3<£ - 2) 



ei7 V4-e 2 " +2 (30-2)V \2 - e?(3<£ - 2) / V</» 



> 1. 



For this we need only show that, for the last two products of fractions, the ratio 
of the second to the first is greater than 1. It is in fact equal to 

- e 2 " +2 (30 - 2) 2 \ /4e 2 - e 2 "(30 - 2] ' 



A - e 2n+2 (30 - 2) 2 / V 4e f - e i™( 3 ^ - 2 ) S 
This is greater than 1 if and only if 

4e 2 -e 2 ™(30-2) 2 4e 2 - e 2 2 n {Z(j) - 2) 2 



s 2 ™ +2 (30-2) 2 4-e 2n+2 (30-2) 2 ' 

Since 2/3 < <f> < 02, we have ef < e\ by Table 1. We divide both numerators 
and denominators by 4 and apply Lemma [5] 



< 



Finally, it remains to show that the functions defining the curves in (161 



(19) are positive above, and negative below, the curves. Consider G nj „_2 for 



even n > 4. Notice that G„^„_2 is positive, 0, or negative according to whether 
'e 2 Y~ 2 (4- e? (n_1) (3^ - 2) 2 \ (2 - e™- 2 (3<f> - 2) 



e x ) U-e 2 2 (n - 1) (30-2) 2 J V2-er 2 (30-2 



(28) 



is > 1, = 1, or < 1. It is therefore enough to show that (|28j) is increasing in 
£ (2/3,02) for each p g (0,1). This follows by showing that the product of 
the first and third factors is increasing and the second factor alone is increasing 
(the fourth factor is constant). The other functions, E n _%, i?„.„- 2 , and H n n _2, 
are treated similarly. □ 

We will later need the following consequence of the proof. 



Proposition 7. Using the notation (12)-(15), if E nn ^ 2 < and E n > for 



some even n > 4, then E n ^ m < for m = 0, 1, . . . , n — 1. If G ni „_2 < and 
H n ,n-2 > for some even n > A, then G„. m < for m = 0, 1, . . . , n — 1 and 
H n ,m < for m = 0, 1, . . . , n — 3. 
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5 Asymptotic stability of limit cycles 



In this section, we consider the case in which the discrete dynamical system 
starts from an arbitrary initial state. We investigate its asymptotic behavior for 
parameters (p, 0) belonging to (0, 1) x (2/3, 1]. 

Let (xq, x±, x 2 ) G be the initial state. (If the initial state is in A^, 
the trajectory will enter A^ eventually by Theorem [2j with an exception as 
noted in that theorem.) Define (2/0,2/1,2/2) := {x 0, X\, X2)Pa, (zq,zi,z 2 ) := 
(2/0, 2/i, 2/2 )-Pb, and (io , u>i, u/2) := (z , zx, z 2 )P B . We know by Proposition [I] 
that (2/0,2/1,2/2) G Ab- In order that (zq, £1,22) G As, we need zq < ttq. Now 
zq 7Tq = a-ix + flixi — 7x, where 

a x := (30 - 2)[-(l + p) + 0(2 + p)]/[2(l + p)] > 0, 
Pi :=0(30-2)(l-p)/[2(l + p)] >0, 

= (l + p)(l + p 2 )-0(3 + p)(l + p + p 2 ) + 30 2 (l + p + p 2 ) 

71 ' 2(l + p)(l + p + p2) 

and so (zq, Z\, z%) G A^ is equivalent to a\x^ + ftiXi < 71. Since 

7l_ 0[l-3(l-0)p](l-p 2 ) 

ax ^° 2(30 -2)(l + p + p2)[-(l + p) + 0(2 + p)] ' 

the region {(xq,Xi 1 x 2 ) G A^ : a>iXo + fi\X\ < 71} is nonempty. 

Next, in order that (100,101,102) G Aa as well, we need wq > ttq. Now 
wo - 7T = -a 2 x - p 2 xi + 72, where 

a 2 := (30 - 2)[(1 + p) 2 (l + p 2 ) - 20(1 + p)(2 + p)(l + p 2 ) 

+ 2 (4 + 5p + 3p 2 + 5p 3 + p 4 )]/[2(l + p) 2 (l + p 2 )] > 0, 
k := (30- 2) 2 0(l-p)/[2(l + p)] >0, 

72 := [-(l + p) 2 (l + p 2 ) 2 + 0(l + p)(5 + p)(l + p 2 )(l + p + p 2 ) 
- 20 2 (l + p 2 )(5 + 5p-p 2 )(l + p + p 2 ) 
+ 3 (1 + p + p 2 )(7 + 5p + 3p 2 + 5p 3 - 2p 4 )] 
/[2(l + p) 2 (l + p 2 )(l + p + p 2 )], 

and so (wq, wi,w 2 ) G A^ is equivalent to a 2 xg + j3 2 x\ < r y 2 . However, 72 is not 
necessarily positive. More importantly, the set of (p, 0) G (0,1) x (2/3,1] for 
which 

^ - ttq = (1 - p)0[-(l + p) 2 (l - 5p)(l + p 2 ) - 120p(l + p) 2 (l + p 2 ) 
+ 2 (2 + lip + 20p 2 + 16p 3 + 16p 4 + 7p 5 )] 
/{2(30-2)(l + p + p 2 )[(l + p) 2 (l + p 2 ) 

- 20(1 + p)(2 + p)(l + p 2 ) + 2 (4 + 5p + 3p 2 + 5p 3 + p 4 )]} 

>0, 

is the region for which {(xo, Xi, x%) G A^ : a 2 xo + f3 2 x\ < 72} is necessarily 
nonempty, which includes (but is not equal to) the union of regions 1-12 in 
Figure [2] below. 



19 



Let us assume that (p, <fi) belongs to this region. Then 

A ABBA = {(x , x 1) x 2 ) £ A A : aix + /3]Xi < 71, a 2 x Q + f3 2 x 1 < 72} 

is nonempty, that is, there exists (xq, x\,x 2 ) £ A A such that the greedy strategy 
begins with the game sequence ABBA. If we solve the system of equations 

aix + P1X1 = 71 and a 2 x + /3 2 Xi = 72, (29) 

simultaneously, we find that x — tto — —0(1 — p) 2 /[2(30 — 2)(1 + p + p 2 )] < 0, 
hence the two lines intersect only outside p(A^) and only one of the inequalities 
aixo + j3\Xi < 71 and a 2 xo + j3 2 x\ < 72 is needed to define p(A ABBA ). This 
tells us that p{A ABBA ) is the intersection of the triangular region {(xo,Xi) : 
Xq > 7Toi x i > 0, Xq + X\ < 1} with exactly one of the two triangular regions 

{(x , xi) : x > tt q , xi > 0, axx + /3%Xi < 71}, 
{(x , x^ : x > tt q , xi > 0, a 2 x + P 2 X\ < 72}, 

hence its closure is convex with at most four extreme points. 

Now the lines (29) intersect the vertical line xq = ttq at xi = a\ := (71 — 
ai7To)//3i > and x\ = a 2 := (72 — a 2 iro)/f3 2 > 0, and they intersect the 
horizontal axis x± = at xq = b\ := 71/ai > no and xq = b 2 := ^ 2 la 2 > ttq. 
The quantities a\,a 2 ,bi,b 2l which are functions of p and (/>, play an important 
role in what follows. In particular, they allow us to partition the set of all 
(p, 4>) £ (0,1) x (2/3,1] such that G^, 2 > into 12 regions, as indicated in 
Figure [2j 

To be more precise, region 1 is d t2 > 0, a 2 < 1 — n , and <j> < (3/4) (1 — p) + 
(2/3)/?; region 2 is a 2 > 1 - tt , b 2 < 1, and <j> < (3/4) (1 - p) + (2/3)p; region 3 
is b 2 > 1; region 4 is 62 < lj <f> < 4>3i an d 4> > (3/4)(l — p) + (2/3)p; region 5 
is > (/) 3 and b\ > 1; region 6 is b\ < 1, a 2 > 1 — 7r , and b 2 < bi; region 7 is 
<22 < 1 — 7To and < 1 — p/3; region 8 is cf) > 1 — p/3 and 62 < b\; region 9 is 
b 2 > &i and < 1 — p/3; region 10 is > 1 — p/3 and 02 > 1 — tto; region 11 is 
b 2 >bx,a 2 <l- 7T , and a>if + fi x h < 7i> where (/ , fi,h) ■= (1, 0, 0)P A P B ; 
and region 12 is a%fo + Pifi > 71 ■ 

We believe that, whenever G4.2 > 0, there is a unique (asymptotically stable) 
limit cycle of the form [1,2]. However, we can prove this only in regions 3, 9, 
10, and 11. 

Theorem 8. For (p, <f>) belonging to region 3 of Figure [1| there is a unique 
limit cycle, which is asymptotically stable and of the form [1,2], as well as an 
unstable equilibrium. Indeed, 

&a = A^ss, (30) 



and for initial states in Ag — A-g (see t^ay), the trajectory eventually enters A A . 

For (p, 4>) belonging to region 9, 10, or 11 of Figure^ there is a unique limit 
cycle, which is globally asymptotically stable and of the form [1,2]. Indeed, 

Aabba = A^gg, A A - A ABBA = A AB7[W g, (31) 

and for initial states in A B , the trajectory eventually enters A A - 
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0.0 0.2 0.4 0.6 0.8 1.0 



P 

Figure 2: Partition into 12 regions of the set where [1,2] is (conjectured to 
be) the form of the unique (asymptotically stable) limit cycle. If the curves are 
labeled 1-9 according to their 0- values at p = 0.2 from smallest to largest, curve 
1 is — (> above), curves 2 and 7 are a-i = 1 — 7r (> between), curves 3 
and 4 are 6 2 = 1 (> between), curve 5 is <p = <fe, curve 6 is b\ — 1 (> below), 
curve 8 is a\ = 1 — ttq (> below) or <j> = 1 — p/3, and curve 9 is 62 = &i (> 
above) or <j> = fo. The curve separating regions 11 and 12 is «i/o + = 71 
(> above), where (/c/1,/2) : = (1, 0, 0)PaJ°b- The regions are defined more 
precisely in the text. 
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Remark. The theorem includes the case (p,(j>) = (1/3,1) studied by Van den 
Broeck and Cleuren [TS]. Results for the regions not covered by the theorem 
will be stated later in the form of a conjecture. 

Proof. Suppose we can show (with some exceptions in the case of region 3) that 
the trajectory eventually reaches A ABB . Then we have three linear discrete 
dynamical systems 

(xo(3m + 3), xi(3m + 3), x 2 (3m + 3)) = (xo(3m), Xi(3m), x 2 (3m))PAP B P B , 
(x (3m + 4), xi(3m + 4), x 2 (3m + 4)) 

= (x (3m + 1), Xl {3m + 1), x 2 (3m + 1))P B P B P A , 
(xo(3m + 5), Xi(3m + 5), x 2 (3m + 5)) 

= (i (3m + 2), aix (3m + 2), x 2 (3m + 2))P B P A P B , 

or 

(x (3m),x 1 (3m),x 2 (3m)) = (x ,x 1 ,x 2 )(PaPbPb)" 1 
(x (3m + l),xi(3m + l),x 2 (3m + 1)) = (x , Xi, x 2 )(PaPbPb) 
(x (3m + 2),xi(3m + 2),x 2 (3m + 2)) = (x , x u x 2 )(P A P B P B ) m P A P Bl 

with (x ,xi,x 2 ) e ^abb- Now (PaPbPb) 171 -> n^^], where IIp.,2] is the 3 x 3 
matrix with each row equal to irti z] , so the limits in the last group of equations 
are Tn 2 ], 7T[i,2]-Paj an d ^[i.2]P A Pbi regardless of the initial state in A ABB . 
This leads to the asymptotic stability. 

Recall that A ABBA is the intersection of three triangular regions. This 
intersection is itself triangular if min{ai,a 2 } < 1 — ttq and min{&i,6 2 } < 1 or 
if min{ai,a 2 } > 1 — ttq and min{6i,6 2 } > 1. And the intersection is four- 
sided if min{ai,a 2 } > 1 — ttq and min{& 1; 6 2 } < 1 (min{ai,a 2 } < 1 — ir and 
min{6i,6 2 } > 1 is impossible; see Figure[2]). 

If for some (p,<j>), PaP b maps the closure of A ABBA into A abb A, then, 
for this (p,4>), A abb A = A ABB . Now PaP b is linear, so PaP b maps the 
closure of Aab ba into Aab ba if and only if it maps the three or four vertices 
of the closure of A abb A into A abb a- in one important case, this completes the 



,mp 



'A, 



proof. If A abb A = A a, then we conclude that (30) holds. This happens when 
a\ > 1 — 7To, a 2 > 1 — ttq, bi > 1, and b 2 > 1. Actually, the last inequality implies 
the first three and holds precisely in region 3 of Figure [2] Recalling the result 
of Theorem]2] we conclude that the assertions about region 3 are established. 
See Figure [3) 



Next, we generalize lines (29). Given an initial state (xq,Xi,x 2 ) G A a, let 
us define z (n) := (x , x\, 1 — xo — xi)PaP b (1, 0, 0) T for each n > 1. Then, for 
n odd, Zo(n) < ttq if and only if a n xo + f3 n x\ < j n , and, for n even, Zo(n) > ttq if 
and only if a n xo + j3 n x\ < 7„, where the coefficients can be expressed in terms 
of the spectral representation of the matrix P B : 

a n := (-l)"- 1 (l,0,-l)P yl Pi J l (l,0,0) T 

= (-1)"(30 - 2)^(1 + p 2 + S) - e?(l + p 2 - 5)1/(45), 
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Figure 3: Two of the four cases of Theorem [8] illustrated with p = 1/3. 



Pn := (-l)™- 1 ^, 1, -1)PaPS(1,0 > Q) t = (-1)"(30 - 2){e% - e?)(l + p 2 )/(2S), 
7 „ := (-l)"- 1 ^- (0,0,1)P4P£](1,0,0) T 

= (-l) n {ea [0(1 + P + P 2 )(3 + 3p 2 + 5) - (1 + p 2 )(l + 2p + 3p 2 + S)] 

- e"[0(l + p + p 2 )(3 + 3p 2 - S) - (1 + p 2 )(l + 2p + 3p 2 - 5)]} 
/[4(l + p + p 2 )5]. 

Notice that these definitions are consistent with (ai,/?i,7i) and (o%/32,72) 
defined earlier. 

The line a n xo + fi n xi = In in the plane intersects the line xq = ttq at 
Xi = a n := (7„ — a„7To)//3 n ; it intersects the line x\ = at xq = 6„ := 7 n /a n ; 
and it intersects the line +xi = 1 at xo = c„ := 

(ln-Pn)l(u n -Pn)- Each of 
the lines a n XQ + fi n x\ — 7„ (n. > 1) passes through the point ((<$> — 27To)/(3</> — 
2), {(f)— 27Ti)/(30— 2)), which lies to the left of the line a; = 7r because 7r > 1/3. 

We turn next to region 9 of Figure [2j This region is determined by 62 > 61 
(equivalent to <f> > </> 2 ) and <Zi > 1 — 7r , from which it follows that &i < 1 (see 
Figure [2| . Consequently, A^ — Aab ba is the triangular region with vertices 
(1,0,0), (6i,0,l — 6i), and (ci,l — ci,0). We claim that (a) PaPb maps these 
three points, and hence the triangular region they determine, into Aabba- 
Further, we claim that (b) PaPb maps the four corners of the closure of Aabba, 
namely (tt o ,0, 1— x a ), (61, 0, l-&i), (c lt 1-Ci,0), and (tt , 1— 7t ,0), into A A bba, 
hence maps the closure of Aabba into Aabba- This is enough to show that, 
starting from A a, pattern ABB repeats forever, possibly after an initial AB. 
To verify (a) and (b), we let 

(/0,/1,/a) := (1,0,0)P a Pb, 
(50,31,52) := (6i,0, 1 - bx)P A P B , 
(ho, hi, ha) := (c x , 1 - ci,0)P a Pb, 
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(r ,ri,r 2 ) := (tt ,0, 1 - ttq)P a Pb, 
(s , Sl ,s 2 ) := (6 1 ,0,1-& 1 )P A P|, 
(io.ii, i 2 ) := (ci,l-ci,0)P A Pi, 
(w ,mi,u 2 ) := (tto, 1 - n Q ,0)P A Pg. 

First, since b 2 > fei in region 9, it follows that if (xq, x\,x 2 ) S A satisfies xq > 7To 
and Qi^o + ftiXi < 71, then (xq,x\,x 2 ) G A^bsa- Thus, it suffices to check 
these two inequalities for each of the seven points listed. We have done this 
algebraically but omit the details, as it is reasonably straightforward. 

We finally consider regions 10 and 11 of Figure [2j which can be treated 
simultaneously. The regions are determined by «i < 1 — ttq (equivalently, 
4> > 1 — p/3), b 2 > bi (equivalently, <fi > 4> 2 ), and ai/o + fiifi < 71, where 
(/c/1,/2) := (1,0,0)PaPb, from which it follows that a\ < 1 — ir and 
61 < 1. Consequently, A a — A abb A is the four-sided region with vertices 
(1, 0, 0), (61, 0, 1 — 61), (tt , <Zi, 1 — 7To — ax), and (ir , 1 — 7r , 0), and the closure of 
A abb A is the triangular region with vertices (61, 0, 1 — b%), (ir Q , a%, 1 — 7Tq — di), 
and (7To, 0, 1 — 7To). As with region 9, it suffices to show that each of the following 
seven points (xq, x\, x 2 ) satisfies the inequalities xq > ttq and a\Xo + P\X\ < 71 
determining A abba- 



ifo, fx, $2) 

(90,91,92) 

(h 0l h 1 ,h 2 ) 
(io,k,i2) 
(so,si,s 2 ) 

(t(),tl)i2) 

(u ,ux,u 2 ) 



= (1,0,0)P a Pb, 

= (b x ,%\-b x )P A P B , 

= (^o,a\, 1 - tto - ai)P A PB, 
= (7ro,l-7ro,0)P 4 P B! 

= (&i,0,l-&i)P A P§, 

= (7T ,ai, 1 - 7T - ai)P4Pfl, 
= (7T0,0,l-7r )P4P|- 



Again, we have done this algebraically but omit the details. See Figure [3j □ 

Let us describe what appears to happen in each of the eight regions not 
covered by Theorem [5J 

Conjecture 9. If (p,4>) belongs to region 1 of Figure^ then there is a unique 
limit cycle, which is asymptotically stable and of the form [1,2], as well as an 
unstable equilibrium. Indeed, 

00 00 

^■A = [J ^(ABBBBABB) k ABB U U ^ABB(ABBBBABB) k ABB> 
fc=0 k=l 

and if the initial state is in As ~ A-g (see the trajectory eventually enters 
A A - 

For (p, <fi) belonging to region 2 of Figure [1| there is a unique limit cycle, 
which is asymptotically stable and of the form [1,2], as well as an unstable 
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equilibrium. Indeed, Aa — A ABB U A ABBBB ABB , and for initial states in 
Ab - Aj, the trajectory eventually enters Aa- 

If(p,4>) belongs to region 4, 5, 6, 7, or 8 of Figure^ then there is a unique 
limit cycle, which is asymptotically stable and of the form [1,2], as well as an 
unstable equilibrium. Indeed, 

oo 

A A = A AB U (J A 



AB k ABB • 



fc=l 



(Note that A AB2 ABB — A ABB .) If the initial state is in Aj- A-g — A BAB — 
A BBAB , the trajectory eventually enters A a — A AB . 

If (p, </>) belongs to region 12 of Figure^ then there is a unique limit cycle, 
which is globally asymptotically stable and of the form [1,2]. Indeed, 

oo 
k,l=0 

and only finitely many of the sets comprising the union are nonempty. If the 
initial state is in As, the trajectory eventually enters Aa- 

The lower boundary of region 1 is the curve G4.2 = 0. In the unshaded 
portion of Figure [2] (where G^ < and <p > 2/3), we have at least one limit 
cycle, as shown by Theorem [6j See the remark following the statement of the 
theorem. Of course, the theorem does not imply that the limit cycles identified 
there are the only ones, but we conjecture that this is in fact true. 

Conjecture 10. Let n > 4 be even. The curve b n ~2 — To — ^ es below 
H n ,n-2 = and above G n+ 2.n — 0, and the function 6„_2 — tt is positive above, 
and negative below, the curve. 

If (p, cj)) satisfies G n .n—2 < and E n -2 > 0, then there are precisely two 
limit cycles, which are of the forms [1, n, 1, n — 2] and [1, n — 2], as well as an 
unstable equilibrium. Indeed, 



A a = A AB „ AB „- a U Q A ( 

00 

U A AB^- 2 AB"AB"- 2 U U A AB™- 2 (AB"AB"- 2 ) k AB™- 2 > 
k=l 

and if the initial state is in As — A^, the trajectory eventually enters A a ■ 

If (p, cj>) satisfies E n -2 < and E n ^ n _2 > 0, then there is a unique limit 
cycle, which is asymptotically stable and of the form [l,n, l,n — 2], as well 
as an unstable equilibrium. Indeed, Aa — A Agrl AB „-2 U ^AB n ~ 2 AB n AB n - 2 ^ 
A AB n AB n AB n - 2 ' an< ^ the initial state is in Ab — A-g, the trajectory eventually 
enters A a- 

If (p, (j)) satisfies £ , nj „_2 < and if„ jn _2 > 0, then there are precisely two 
limit cycles, which are of the forms [l,n, l,n— 2] and [l,n], as well as an unstable 
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equilibrium. Indeed, 

oo 

= ^AB"AB"- 2 U U ^{AB n AB n - 2 ) k A~B™ : U ^AB n ^ 2 AB n AB n ^ 2 
k=0 

oo 

U [J ^AB^- 2 {AB"AB"- 2 ) k AB^ U ^AB n AB n AB n ^ 2 ' 
fc=0 

and z/ i/ie initial state is inAj- A-g , i/ie trajectory eventually enters Aa ■ 

If (p, 4>) satisfies H n>n _2 < and fo„_2 — tto > 0, i/ien i/iene is a unique 
limit cycle, which is asymptotically stable and of the form [l,n], as well as an 
unstable equilibrium. Indeed, 

oo oo 

A A = [J A(AB"AB™- 2 ) k AB^ U U ^AS«- 2 (AB"AB"- 2 )'=AB^' 
fc=0 k=0 

and if the initial state is in — A-g , the trajectory eventually enters Aa ■ 

If (p, 4>) satisfies b n -2 — < and G n +2,n > 7 i/ien i/iere is a unique 
limit cycle, which is asymptotically stable and of the form [l,n], as well as an 
unstable equilibrium. Indeed, 

oo oo 

Aa = [J ^(AB"-+ 2 AB") k AB^ U U ^AB"(AB"+ 2 AB") k AB^> 
k=0 k=l 

and if the initial state is in Ab — A-g, the trajectory eventually enters A a ■ 

6 Asymptotic cumulative average profit 

In Section [T] we stated that, if game B is eventually played forever, we have an 
asymptotically fair game, whereas if the pattern of games is eventually periodic, 
we have an asymptotically winning game. Here we try to justify these assertions. 
In Section 4 we found that the periodic patterns [l,n] for even n > 2 and 
[1, n, 1, n — 2] for even n > 4 can occur. If our Conjectures [9] and 10 are correct, 
then these are the only periodic patterns that can occur. 

Theorem 11. We denote the asymptotic cumulative average profit per game 
played by p-g in the situation where game B is eventually played forever, by p[i, n ] 
in the case of a limit cycle of the form [1, n] for even n > 2, and by P[i in ,i,n-2] 
in the case of a limit cycle of the form [1, n, 1, n — 2] for even n > 4. Then all 
(Cesdro) limits exist, and p-g = 0, M[i,nl > 0; an< ^ A t [i,n,i,n-2] > 0. 

Remark. This shows that the greedy strategy exhibits the Parrondo effect when 
<fi > 2/3 (with some exceptions) but not when <f> < 2/3. 

Proof. In the situation where game B is eventually played forever, p-g = (/>7r£, 
where 7r is the stationary distribution of Pb and C := (2po — 1, 2pi — 1, 2pi — 1) T 
withpo = P 2 1 (1+P 2 ) andpi = l/(l+p). We note that, for given (zq, z\, z^) G A, 



26 



(zq, z l ,z 2 )C = z {2p - 1) + (1 - z )(2pi - 1) = z (2p - 2pi) + 2pi - 1. Hence 
7r£ = 7To(2po — 2pi) + 2pi — 1 = 0, where the second (algebraic) step will be 
used again below. Thus, = 0. 

In the case of a limit cycle of the form [1, n] for even n > 2, 

/*[l,n] = j-^-ff [1 , n] P J i(J + P B + • • • + Pg-^C (32) 

Now 

7r [lin] P A P^C = 7r [1 ,„ ] P 4 P^(l,0 ! 0) T (2p - 2 Pl ) + 2 Pl - 1 
= (tt + E n , m /D n )(2p - 2 P i) + 2pi - 1 
= ( J E n , rn /£> n )(2po-2p 1 ), 



so ( 32 ) can be written as 

20 , . ^ E, 



n. m 



A, 



By Proposition [7J £? n , m < for m = 0, 1, . . . ,n — 1. Since p < pi, it follows 
that P[i, n ] > 0. 

In the case of a limit cycle of the form [1, n, 1, n — 2] for n > 4 even, 

M[l,n,l,n-2] = ^-T[l,n,l,n-2] [Pa(^ + Pb H + Pfp 1 ) 

+ P A PgP 4 (J + P B + --- + P5- 3 )]C. (33) 



Now 



and 



f[i, B ,i,MWf = ^li.n.i.^lWll, 0, 0) T (2p - 2pi) + 2 Pl - 1 
= (tto + G n , m // n )(2p - 2pi) + 2pi - 1 

= (G„,m//n)(2p -2pi) 
7T[l,r 1 ,l,n-2]P4PBP4PB l C 

= 7r [1) „ il)n _ 2] P A PgP4Pg 1 (l ) 0, 0) T (2p - 2 Pl ) + 2pi - 1 
= (tt + ff n , m //„)(2p - 2p0 + 2pr - 1 

= (H ntm /I n )(2po ~ 2pi), 



so ( 33 ) can be written as 

M[l,n,l,n-2] = "bo - Pi) 2^ + -IPO - Pi) 2^ "T" ' 

m=0 ™ m=0 ™ 

By Proposition [TJ G„, m < for m = 0, 1, . . . , n — 1 and H n>m < for m = 
0, 1, ... ,n — 3. Since po < pi, it follows that /ih )n , i n -2] > 0. □ 
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